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SUMMARY  AMD  CONCLUSIONS 


Problem 


The  Naval  Preparatory  School  (NPS)  serves  both  as  a  preliminary 
training  program  to  prepare  enlisted  candidates  for  officer  training, 
and  as  a  situational  screen  to  eliminate  those  whose  academic  capac¬ 
ity  is  inadequate.  Much  concern  has  been  expressed  over  the  high 
attrition  rate  of  enlisted  men  selected  for  NPS,  and  among  NPS  gradu¬ 
ates  at  the  Naval  Academy.  The  Naval  Personnel  Research  and  Develop¬ 
ment  Laboratory  is  responsible  for  investigating  the  effectiveness  of 
the  NPS  system,  particularly  with  regard  to  selection  of  enlisted 
candidates  for  NPS,  and  eventual  acceptance  by  the  Naval  Academy. 

The  objective  of  the  investigation  is  to  assess  and  improve  the  ef¬ 
fectiveness  of  the  current  NPS  selection  procedures. 

> 

Background  and  Requirements  ’ 

Hie  mission  of  NPS  is  to  provide  intensive  instruction  designed 
to  prepare  potential  U.  S.  Naval  Academy  (USNA)  applicants  from  the 
Regular  and  Reserve  Navy  and  Marine  Corps  with  the  requisite  skills 
for  matriculation  in,  and  satisfactory  completion  of,  the  USNA  cur¬ 
riculum. 

Although  most  of  the  attrition  occurs  eirly  from  both  NPS  and 
USNA,  it  represents  an  estimated  annual  loss  to  the  Navy  of  between 
one  and  three  million  dollars. 

In  order  to  reduce  these  financial  losses  and  to  aid  in  the  pro¬ 
curement  of  an  adequate  number  of  well-trained  Naval  Officers,  the 
Assistant  Chief  for  Education  and  Training  (Pers-C)  requested  the 
Assistant  Chief  for  Plans  and  Programs  (Pers-A)  to  initiate  the 
present  study. 

Approach 

The  predictor  variables  were  all  measures  of  scholastic  ability 
and/or  achievement.  The  operational  predictors  consisted  of  five 
measures  of  high  school  academic  performance  and  the  School  and  Col¬ 
lege  Ability  Tests.  These  seven  variables  were  combined  into  the 
Current  NPS  Selection  Composite.  Four  experimental  predictors  were 
also  examined:  the  English  and  mathematics  achievement  tests  of  the 
College  Entrance  Examination  Board;  and,  the  verbal  and  mathematics 
sections  of  the  Scholastic  Aptitude  Tests.  Two  criteria  were  employed 
NPS  Final  Grade  Average  (FGA) ;  and,  USNA  Selection  vs.  Rejection  (S/R) 

Trie  applicant  sample  consisted  of  2„46  persons  eligible  for  the 
1966-67  NPS  program.  The  selectee  sample  consisted  of  those  113 
persons  accepted  by  NPS. 
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Correlation  and  regression  analyses  were  performed  against  each 
of  the  two  criteria  for  the  Current  NPS  Selection  Composite  and  the 
operational  and  experimental  predictors  described  above,  taken  indi¬ 
vidually  and  in  various  combinations. 

Mai or  Findings 

1.  The  predictive  validities  of  the  Current  NPS  Selection  Com¬ 
posite  (FGA:  r.  ■  .643,  and  S/R:  r^  -  .402)  arc  moderately  high, 

2.  The  predictive  validities  of  the  optimally-weighted  combi¬ 
nations  of  operational  predictors  (FGA:  -  . V  3 3 ,  and  S/R»  gR  -  ,398) 

are  higher  than,  or  equal  to,  those  of  the  Current  NPS  Selection  Com¬ 
posite,  while  requiring  less  administrative  time  and  effort. 

3.  The  predJ.ctive  validities  of  the  optimally-weighted  combi¬ 
nations  of  operational  and  experimental  predictors  (FGA:  R  ■  .871, 

and  S/R:  R  -  .655)  arc.  higher  than  those  based  upon  the  {jest  combi¬ 
nation  of  operational  predictors,  and  are  considerably  higher  than 
those  of  the  Current  NPS  Selection  Composite,  while  requiring  less 
administrative  time  and  effort. 

Conclusions 


1.  The  optimally-weighted  composites,  especially  those  based 
upon  operational  and  experimental  predictors,  are  more  efficient  than 
the  Current  NPS  Selection  Composite. 

2.  Although  the  multiple  validity  coefficients  reported  above 

(  R)  have  been  corrected  for  shrinkage,  some  additional  reduction  in 
the  magnitude  of  the  coefficients  is  expected  when  these  optimally- 
weighted  composites  are  employed  in  another  NPS  applicant  6airole. 
However,  it  seems  likely  that  these  optimally-weighted  composites 
will  sustain  their  advantage  over  the  Current  NPS  Selection  Composite 
in  predictive  validity. 

Recommendations 


1.  Data  should  be  required  for  all  eligible  NPS  applicants  on 

the  following  tests:  (a)  School  and  College  Ability  Tests;  (b)  English 
and  mathematics  achievement  tests  of  the  College  Entrance  Examination 
Board;  and,  (c)  Scholastic  Aptitude  Tests.  (Pages  6-9) 

2.  The  set  of  optimal  composites  based  upon  operational  and  ex¬ 
perimental  predictors  should  be  adopted  as  a  replacement  for  the  Cur¬ 
rent  NPS  Selection  Composite.  This  should  improve  the  "batting 
average"  of  selection  decisions,  when  used  in  conjunction  with  other 
information  considered  pertinent  by  the  NPS  Selection  Board.  (Page  9) 


3.  Follow-up  studies  are  required,  as  the  necessary  data  become 
available,  to:  (a)  cross-validate  the  results  reported  herein;  and 
(b)  to  validate  the  predictors  against  more  ultimate  criteria  of  suc¬ 
cess  like  performance  in,  and  graduation  from  the  USNA  program,  and 
subsequent  on-the-job  performance.  (Page  10) 

4.  Additional  recruitment  efforts  would  help  to  insure  the 
improvement  of  the  current  selection  ratio,  thereby  making  it  possible 
to  take  maximum  advantage  of  the  predictive  validity  of  the  composites. 
(Pages  6-9) 

5.  The  procedures  involved  in  computing  and  using  any  of  the 
alternative  selection  strategies,  while  rot  complex,  are  tedious. 

In  order  to  minimize  the  time  and  effort  required  of  NPS  ad.  '.nis- 
trative  personnel,  development  and  utilization  of  EQP  procedures  . -1 
mandatory.  Once  operational,  computer-assisted  selection  would 
enable  the  NPS  Selection  Board  to  focus  on  the  background  character¬ 
istics  of  applicants  which  are  difficult  to  quantify.  (Pages  B-l  - 
B-12) 


THIS  PAGE  IS  BI.ANK 


1 


REPORT  USE  AND  EVALUATION 


Feedback  from  consumers  Is  a  vital  element  in  improving  products 
so  that»they  better  respond  to  specific  needs.  To  assist  the  Chief  of 
Nfeval  Personnel  in  future  planning,  it  is  requested  that  the  use  and 
evaluation  form  on  the  reverse  of  this  page  be  completed  and  returned. 
The  page  is  preaddressed  and  franked;  fold  in  thirds,  seal  with  tape, 
and  mail. 


Postage  and  Fees  Paid 
Navy  Department 

Official  Business 


Commanding  Officer 

Naval  Personnel  Research  and  Development 
Laboratory,  Bldg.  200 
Washington  Navy  Yard 
Washington,  D.  C.  20390 


Report  Title  &  No. : 

PREDICTION  OF  SUCCESS  IN  THE  U.  S.  NAVAL  PREPARATORY  SCHOOL 
WRR  70-7 

1.  Evaluation  of  Report.  Please  check  appropriate  column. 


FACTORS 

RATING 

LOW  AVE  HIGH 

COMMENTS 

Usefulness  of  Data 

Timeliness 

Completeness 

Technical  Accuracy 

Validity  of  Recommen¬ 
dations 

Presentation  &  Style 

Other 

■ 

■ 

■ 

■ 

■ 

■ 

i 

■ 

■ 

■ 

2.  Use  of  Report  Please  fill  in  answers  as  appropriate. 


a.  What  are  your  main  uses  for  the  material  contained  In  the  report? 

b.  What  changes  would  you  recommend  in  report  format  to  make  it  more  useful  ? 

o.  What  types  of  research  would  be  moat  useful  to  you  for  the  Chief  of 
Naval  Personnel  to  conduct? 

d.  Do  you  wish  to  remain  on  our  distribution  list? 

e.  Please  make  any  general  comments  you  feel  woula  be  helpfhl  to  us 
in  planning  our  research  program. 

NAME: _ CODE: _ 

ORGANIZATION: _ 

ADDRESS: 


gpo 


TABLE  OF  CONTENTS 


Page 

Foreword . 11 

SUMMARY  AND  CONCLUSIONS .  ill 

Use  and  Evaluation  Form  < authorized  tear-out  form)  . 

List  of  fables .  ViH 

BACKGROUND .  1 

PURPOSE . 1 

METHOD  .  2 

Variables .  2 

Operational  Predictor  Variables .  2 

Experimental  Predictor  Variables  .  3 

Selection  Composite .  3 

Criterion  Variables.  .  .  .  3 

Samples .  4 

Applicant  Sample  .  4 

Selectee  Sample .  4 

Data  Analyses .  4 

Correlation  Analyses  •  .  * 

Regression  Analyses . 4 

RESULTS  AND  INTERPRETATION  .  5 

Validity  of  the  Current  NPS  Selection  Composite.  ....  5 

Validity  of  the  Operational  Predictors  .  5 

Validity  of  the  Experimental  Predictors .  6 

Multiple  Validity  of  the  Operational  Predictors.  ....  6 

Multiple  Validity  of  the  Operational  and  Experimental 

Predictors . 9 

Comparison  of  Alternative  Prediction  Composites .  9 

Predictions  for  Future  NPS  Applicants .  9 

DISCUSSION  .  9 

REFERENCES .  11 

Appendix  A  -  Results  of  Correlation  Analyses  .  A-l 

Appendix  B  -  Procedures  for  Predicting  Criterion  Scores.  .  .  B-l 

vii 


TABLE  OF  CONTENTS  (Coat'd) 


Page 

Distribution  List 

DD  Form  1473  (Document  Control  Data  -  R&D) 


LIST  OF  TABLES 

1.  Validity  of  the  Individual  Predictors  and  the  Selec¬ 

tion  Composite  for  NPS  Applicants  .  6 

2.  Components  and  Predictive  Validity  of  the  Alternative 

Selection  Composites  for  Forecasting  NPS  Final 

Grade  Average  for  NPS  Applicants .  7 

3.  Components  and  Predictive  Validity  of  the  Alternative 

Selection  Composites  for  Forecasting  USNA  Selec¬ 
tion  vs.  Rejection  for  NPS  Applicants .  8 

A-l.  Intercorrelations,  Means,  and  Standard  Deviations  of 

All  Predictor  and  Criterion  Variables  in  the  NPS 
Applicant  Sample .  A-2 

A-2.  Intercorrelations,  Means,  and  Standard  Deviations  of 

All  Predictor  and  Criterion  Variables  in  the  NPS 
Selectee  Sample  .  A-3 

B-l.  High  School  Achievement  Data  on  A  Typical  NPS 

Applicant .  B-4 

B-2.  Standardized  Test  Data  on  A  Typical  NPS  Applicant.  .  .  B-6 

B-3.  Optimal  Raw  Score  Weights  for  Predicting  NPS  Final 
Grade  Average  and  USNA  Selection  vs.  Rejection 
for  NPS  Applicants  from  Scores  on  the  Operational 
Predictors . .  B-8 

B-4.  Optimal  Raw  Score  Weights  for  Predicting  NPS  Final 
Grade  Average  and  USNA  Selection  vs.  Rejection 
for  NPS  Applicants  from  Scores  on  the  Operational 
and  Experimental  Predictors  .  B-10 


viii 


BACKGROUND 


Each  year,  the  Secretary  of  the  Navy  may  appoint  to  the  U.  S. 

Naval  Academy  (USNA)  85  enlisted  men  from  each  of  the  following 
sources:  (a)  Regular  Navy  and  Marine  Corps;  and  (b)  Naval  Reserve 

and  Marine  Corps  Reserve.  In  order  to  compete  for  these  appointments, 
all  applicants  in  the  first  group  must  attend  the  U.  S.  Naval  Prepara¬ 
tory  School  (NPS) ,  Bainbridge ,  Maryland.  A  limited  number  of  openings 
in  the  NPS  program  are  available  for  applicants  in  the  second  group 
(10).1 

The  mission  of  NPS  is  to  provide  a  course  of  intensive  instruction 
designed  to  prepare  potential  USNA  applicants  with  the  requisite  skills 
for  matriculation  in,  and  successful  completion  of,  the  rigorous  USNA 
program  (2_,  11)  . 

The  NPS  and  USNA  programs  represent  a  considerable  investment  by 
the  Navy.  For  each  candidate,  the  costs  of  these  educational  programs 
are  approximately  $4,000  and  $34,000,  respectively  (9) .  The  attrition 
rate  between  NPS  matriculation  and  USNA  graduation  is  alarmingly  high. 
Using  average  figures  based  upon  the  USNA  classes  of  1962-1965 ,  onlv 
61%  of  those  selected  for  NPS  successfully  completed  the  program.  An 
average  of  48%  of  the  original  NPS  selectees  were  found  to  be  quali¬ 
fied  for  USNA  entrance  and  only  41%  of  the  original  group  entered  USNA. 
(1) .  Finally,  only  18%  of  the  original  group  of  NTS  selectees  gradu¬ 
ated  from  USNA.  This  represents  a  total  attrition  rate  of  82%.  Despite 
the  fact  that  most  dropouts  occur  early  in  these  educational  programs, 
the  annual  loss  to  the  Navy  occasioned  by  this  high  rate  of  attrition 
is  between  one  and  three  million  dollars  (9). 

In  order  to  decrease  these  financial  losses  and  to  aid  in  the  pro¬ 
curement  of  an  adequate  number  of  well-trained  Naval  Officers,  the 
Assistant  Chief  for  Education  and  Training  (Pers-C)  requested  the  As¬ 
sistant  Chief  for  Plans  and  Programs  (Pers-A)  to  initiate  the  present 
study  (1) . 


PURPOSE 

This  investigation  is  the  first  in  a  series  of  studies  designed 
to  assess  and  improve  the  efficacy  of  the  current  NPS  selection 
strategy.  The  purpose  of  this  report  is  to  inform  a  general  audience 
of  the  results  obtained  to  date.  Another,  more  technical  report  (8) 
describes  the  methodology  and  results  of  the  various  statistical 
analyses  in  greater  detail. 


-  —  j 

All  underlined  numbers  enclosed  by  parentheses  refer  to  corre¬ 
sponding  numbers  of  documents  listed  under  References .  | 
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METHOD 


Variables 


The  name,  abbreviation, 2  and  definition  of  each  of  the  variables 
are  as  follows: 

Operational  Predictor  Variables^ 

Data  on  these  variables  were  available  to  the  NFS  Selection  Board 
prior  to  selection  decisions. 

(1)  Malor  Subject  Unit  Score  (MSU)  -  The  number  of  one  year, 
college  preparatory’  courses  passed  (a  grade  of  C  or  higher)  in  the 
following  subjects:  English;  algebra;  plane  geometry;  solid  geometry; 
trigonometry;  chemistry;  and  physics. 

(2)  Optional  Subject  Unit  Score  (OSU)  -  The  number  of  one 
year  college  preparatory  courses  passed  (a  grade  of  C  or  higher)  in 
the  following  subjects:  Latin;  modern  language;  social  studies;  and, 
sciences  and  mathematics  courses  not  considered  MSU. 

(3)  Major  Subject  Quality  Score  (MSQ)  -  A  score  which  describes 
the  grades  earned  in  the  MSU,  as  follows:  "A"«*4;  "B"=3;  "C"=2 ;  "D"=0; 
and  "F"-0. 

(4)  Optional  Subject  Quality  Score  (OSQ)  -  A  score  which  de¬ 
scribes  the  grades  earned  in  the  OSU,  as  follows:  "A"»4;  "B"=2;  "C"=l; 
"D"“0;  and.  "F"=0 . 

(5)  High  School  Completion  Score  (HSC)  -  A  score  of  15  is  as¬ 
signed  to  high  school  graduates ,  while  non-graduates  receive  a  score 
of  zero. 


(6)  SCAT  Verbal  Ability  Raw  Score  (SVAR)  -  The  number  of 
questions  answered  correctly  on  the  verbal  section  of  the  Cooperative 
School  and  College  Ability  Tests. 

(7)  SCAT  Verbal  Ability  Converted  Score  (SVAC)  -  The  converted 
score  assigned  to  SVAR. 


Abbreviations  for  the  variable  names  will  be  employed  through¬ 
out  the  report. 

3 

Raw  scores  on  the  two  SCAT  variables  were  employed  in  the  Cur¬ 
rent  NPS  Selection  Composite.  However,  only  the  two  corresponding 
converted  scores  were  used  in  the  statistical  analyses  Q,  4) . 
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(8)  SCAT  Quantitative  Ability  Raw  Score  (SQAR)  -  The  number 
of  questions  answered  correctly  on  the  quantitative  section  of  the 
Cooperative  School  and  College  Ability  Tests. 

(9)  SCAT  Quantitative  Ability  Converted  Score  (SQAC)  -  The 
converted  score  assigned  to  SQAR. 

Experimental  Predictor  Variables 

Data  on  these  variables  were  obtained  on  NPS  selectees  about  mid¬ 
way  through  the  academic  year.  Therefore,  this  test  information  was 
not  available  for  use  by  the  NPS  Selection  Board  in  making  selection 
decisions . 

(1)  CEEB  English  Achievement  Score  (CBEA)  -  The  converted 
score  obtained  on  the  English  Comprehension  achievement  test  of  the 
College  Entrance  Examination  Board. 

(2)  CEEB  Mathematics  Achievemer  Score  (CBMA)  -  The  converted 
score  obtained  on  the  Intermediate  Mat':..  ,atics  achievement  test  of 
the  College  Entrance  Examination  Board. 

(3)  SAT  Verbal  Aptitude  Score  (SATV)  -  The  converted  score 
obtained  on  the  verbal  section  of  the  Scholastic  Aptitude  Tests. 

(4)  SAT  Mathematics  Aptitude  Score  (SATM)  -  The  converted 
score  obtained  on  the  mathematics  section  of  the  Scholastic  Aptitude 
Tests . 

Selection  Composite 

This  combination  of  seven  operational  predictors  was  employed  in 
selecting  NPS  applicants  in  the  1966-67  academic  year. 

(1)  Current  NPS  Selection  Composite  (CSC)  -  The  sum  of  MSU, 
weighted  by  two,  plus  the  sum  of  OSU,  MSQ,  OSQ,  HSC,  SVAK,  and  SQAR. 

Criterion  Variables 


Data  on  these  two  variables  were  available  for  all  persons  selected 
for  the  NPS  program. 

(1)  NPS  Final  Grade  Average  (EGA)  -  The  final  grade  average 
obtained  in  the  NPS  program. 

(2)  USNA  Selection  vs.  Rejection  (S/R)  -  A  score  which  ex¬ 
presses  the  U.  S.  Naval  Academy  admission  status:  selectees  were  as¬ 
signed  a  score  of  one,  while  rejectees  received  a  score  of  zero. 
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Samples 


Applicant  Sample 

This  sample  of  246  consists  of  eligible  NPS  applicants  for  the 
1966-67  academic  year.  In  order  to  be  considered  an  "eligible"  ap¬ 
plicant,  a  person  had  to  possess  a  number  of  requisite  qualifications 
delineated  elsewhere  ( 2 ) . 

Selectee  Sample 

This  sample  of  113  consists  of  eligible  applicants  (as  described 
above)  who  were  selected  for  admission  to  NPS,  matriculated,  and  re¬ 
mained  in  the  program  for  at  least  one  marking  period. 


Data  Analyses 


Correlation  Analyses 

Means  (average  score) ,  standard  deviations  (a  measure  of  disper¬ 
sion,  indicating  the  scatter  of  scores  about  the  mean),  and  inter- 
correlations^*  for  the  operational  predictor  variables  were  computed 
in  the  applicant  sample.  Since  data  on  all  applicants  were  not  avail¬ 
able  for  the  four  experimental  predictor  variables  and  the  two  cri¬ 
terion  variables,  the  statistics  involving  these  variables  were  esti¬ 
mated  using  techniques  described  elsewhere  (6.,  pp.  413-417). 

Regression  Analyses 

Multiple  regression  analyses  of  the  operational  predictor  vari¬ 
ables  were  performed  against  each  of  the  two  criteria.  In  addition, 
multiple  regression  analyses  incorporating  the  operational  and  ex¬ 
perimental  predictor  variables  were  made. 


A  correlation  coefficient  describes  the  degree  of  association  or 
linear  relationship  between  two  variables.  A  perfect  relationship 
yields  a  correlation  of  plus  or  minus  one,  while  the  absence  of  any 
relationship  is  indicated  by  a  correlation  of  zero.  If  the  two  vari¬ 
ables  are  directly  related,  the  sign  of  the  correlation  if  positive; 
a  negative  correlation  indicates  an  inverse  relationship. 
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RESULTS  AND  INTERPRETATION5 

The  i  .tercorrelations ,  means,  and  standard  deviations  of  all 
variables  in  the  NFS  applicant  sample  are  presented  in  Table  A-l  of 
Appendix  A.  Table  A-2  presents  the  same  information  computed  in  the 
sample  of  NPS  selectees. 

The  appropriate  weights  for  the  alternative  selection  composites 
and  procedures  for  employing  them  to  estimate  criterion  scores  (FGA 
and  S/R)  for  NPS  applicants  are  explained  in  Appendix  B. 

The  validity  coefficient  (a  correlation  between  a  criterion  and 
a  predictor  or  composite  of  predictors)  constitutes  the  basis  for 
evaluating  the  various  predictors  as  potential  replacements  for  CSC. 

Criterion  data  (FGA  and  S/R)  were  not  available  for  those  persons 
rejected  by  NPS.  The  validity  coefficients  of  the  individual  pre¬ 
dictors  and  CSC  were  computed  in  the  sample  of  NPS  selectees.  Since 
the  selectees  cannot  be  considered  a  representative  sample  of  NPS  ap¬ 
plicants,  the  validity  coefficients  were  corrected  to  provide  better 
estimates  of  the  value  of  the  predictors  for  making  selection  decisions 
in  the  NPS  applicant  group. 

Validity  of  the  Current  NPS  Selection  Composite 

The  best  available  estimate  of  the  validity  of  CSC  for  predicting 
FGA  in  the  NPS  applicant  sample  is  .643,  as  shown  in  Table  1.  The 
corresponding  validity  of  this  composite  for  predicting  S/R  for  NPS 
applicants  is  .402.  These  validity  estimates  constitute  the  standard 
against  which  the  validity  of  the  various  predictors,  taken  singly  and 
in  combination,  can  be  evaluated. 

Validity  of  the  Operational  Predictors 

As  shown  in  Table  1,  SQAC  and  MSQ  are  the  two  most  effective  single 
operational  predictors  for  forecasting  FGA  for  NPS  applicants  (£  =  .615 
and  .590,  respectively).  The  same  two  operational  predictors  are  also 
the  most  predictive  of  S/R  (r  ^  =  .337  and  .349,  respectively). 

None  of  the  operational  predictors,  taken  alone,  is  as  effective 
as  CSC  for  predicting  FGA  (j:  =  .643)  or  S/R  (r  ^  -  .402).  This  finding 
is  not  surprising  since  CSC  utilizes  information  contained  in  seven 
operational  predictors. 


The  multiple  validity  coefficients  have  been  corrected  for 
shrinkage  (  R  )  (^5,  p.  401).  They  represent  the  best  available  es¬ 
timate  of  tfie  validities  to  be  expected  in  future  NPS  applicant 
samples . 
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TABLE  1 


Validity  of  the  Individual  Predictors  and  the  Selection 
Composite  for  NPS  Applicants 
(N-246)a 


Criterion  Variables 

Predictor  Variables 

FGA  S/R 

Operational  Predictors: 


MSU 

.512 

.308 

OSJ 

-.02] 

.012 

MSQ 

.590 

.349 

OSQ 

.447 

.203 

HSC 

.055 

.030 

SVAC 

.263 

.213 

SQAC 

.615 

.337 

Experimental  Predictors: 

CBEA 

.561 

.454 

CBMA 

.730 

.509 

SATV 

.496 

.364 

SATM 

.770 

.623 

Selection  Composite: 

CSC 

.643 

.402 

aThe  validity  coefficients  of  the  individual  predictors  as  well  as 
the  CSC  were  estimated  (j6,  pp.  415-417). 


Validity  of  the  Experimental  Predictors 


The  two  best  experimental  predictors  for  forecasting  FGA  for  NPS 
applicants  are  SATM  (r.  =  .770)  and  C3MA  (r.  =  .730),  as  shown  in  Table  1. 
The  same  two  experimental  predictors  are  also  the  best  predictors  of 
S/R  (r  k  *=  .623  and  .509,  respectively).  They  botn  exceed  CSC  and  all 
the  opirational  predictors,  taken  individually,  in  predictive  validity 
for  both  criteria. 

Multiple  Validity  of  the  Operational  Predictors 

As  shown  in  Table  2,  the  best  combination  of  operational  predictors 
for  forecasting  FGA  consists  of  three  predictors:  MSQ,  OSQ,  and  SQAC. 
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This  combination  of  three  operational  predictors,  when  properly  weight¬ 
ed,  appears  to  be  more  effective  (  R  ■  .733)  than  CSC  (r.  =  .643). 


TABLE  2 


Components  and  Predictive  Validity  of  the  Alternative  Selection 
Composites  for  Forecasting  NPS  Final  Grade  Average 
for  NPS  Applicants® 

(N-246) 


Components  of  the 
Alternative  Selection  Composites 

Predictive  Validity 
for  FGA 

Current 

NPS  Selection  Composite: 

r  * 

.643 

1. 

MSU 

2. 

OSU 

3. 

MSQ 

4. 

OSQ 

5. 

HSC 

6. 

SVAR 

7  % 

SCAR 

Optimal 

Composite  of  Operational 

Predictors : 

R  = 
c 

•  733b 

1. 

MSQ 

2. 

OSQ 

3. 

SQAC 

Optimal 

Composite  of  Operational 

and 

Experimental  Predictors: 

R  = 
c 

•  845b 

1. 

MSQ 

2. 

OSQ 

3. 

CBEA 

4. 

CBMA 

5. 

SATM 

aThese  validity  coefficients  represent  the  best  available  esti¬ 
mates  . 

bThe  multiple  validity  coefficients  have  been  corrected  for 
shrinkage  ( ,R)  (.5,  p.  401).  As  such,  they  represent  the  best  avail¬ 
able  estimate  of  the  validities  to  be  expected  in  future  NPS  appli¬ 
cant  samples. 
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Two  of  these  same,  operational  predictors,  MSQ  and  SQAC,  make  up 
the  best  combination  for  predicting  S/R  for  NPS  applicants  (  R  =  .398), 
as  shown  in  Table  3.  This  represents  about  the  same  predictSve  ef¬ 
ficiency  as  CSC  (x  **  .402). 


TABLE  3 

Components  and  Predictive  Validity  of  the  Alternative  Selection 
Composites  for  Forecasting  USNA  Selection  vs.  Rejection 
for  NPS  Applicants® 

(N-246) 


Components  of  the  Predictive  Validity 

Alternative  Selection  Composites  for  S/R 


Current  NPS  Selection  Composite: 


1.  MSU 

2.  OSU 

3.  MSQ 

4.  OSQ 

5.  HSC 

6.  SVAR 
/ .  SQAR 

Optimal  Composite  of  Operational  Predictors: 


1.  MSQ 

2.  SQAC 

Optimal  Composite  of  Operational  and 
Experimental  Predictors: 


1.  CBEA 

2 .  SATM 


r  -  .402 
pb 


R  -  .398b 


R  «  .633b 


£ 

These  validity  coefficients  represent  the  best  available  esti¬ 
mates  . 

bThe  multiple  validity  coefficients  have  been  corrected  for 
shrinkage  (  R)  (^,  p.  401).  As  such,  they  represent  the  best  avail¬ 
able  estimate  of  the  validities  to  be  expected  in  future  NPS  appli¬ 
cant  samples. 
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It  appears  that  the  availability  and  proper  utilization  of  inform¬ 
ation  on  these  three  operational  predictors  (MSQ,  OSQ,  and  SQAC)  will 
allow  the  accuracy  of  predictions  of  S/R  to  be  approximately  main¬ 
tained  and  will  increase  the  accuracy  of  predictions  of  FGA  for  NPS 
applicants  over  that  afforded  by  CSC,  which  requires  information  on 
seven  operational  predictors. 


Multiple  Validity  of  the  Operational  and  Experimental  Predictors 


The  best  combination  of  operational  and  experimental  predictors 
for  forecasting  FGA  for  NPS  applicants,  as  shown  in  Table  2,  consists 
of  five  predictors:  MSQ,  OSQ,  CBEA,  CBMA,  and  SATM.  This  combination 
of  five  operational  and  experimental  predictors,  when  properly  weighted, 
appears  to  be  considerably  more  effective  (  R  ■  .845)  than  CSC  (jr  ■  .643). 


Two  of  the  experimental  predictors,  CBEA  and  SATM  comprise  the 
most  promising  composite  for  the  prediction  of  S/R  for  NPS  applicants 
(  R  *»  .633),  as  shown  in  Table  3.  While  none  of  the  operational  pre¬ 
dictors  appear  to  enhance  the  prediction  of  this  criterion,  it  should 
be  noted  that  it  appears  to  offer  considerable  improvement  over  CSC 


(£pb 


.402). 


Comparison  of  Alternative  Prediction  Composites 


In  addition  to  CSC,  two  alternative  selection  composites  for  each 
of  the  two  criteria  have  been  discussed:  (1)  the  optimal  composite  of 
operational  predictors;  and,  (2)  the  optimal  composite  of  operational 
and  experimental  predictors. 


Prediction  of  FGA  or  S/R  for  a  specific  individual  applicant  based 
upon  the  various  prediction  composites  will  exhibit  some  variation. 

The  predictions  based  upon  CSC  are  the  least  accurate,  while  predictions 
based  upon  the  optimal  composite  of  operational  and  experimental  pre¬ 
dictors  are  the  most  accurate. 


Predictions  for  Future  NPS  Applicants 

Forecasts  of  FGA  and  S/R  for  a  particular  individual  applicant  to 
a  future  NPS  class  can  be  quite  erroneous ,  regardless  of  the  prediction 
composite  employed.  However,  predictions  for  an  entire  applicant  group 
based  upon  the  optimal  composite  of  operational,  or  operational  and  ex¬ 
perimental  predictors  will,  on  the  whole,  be  more  accurate  than  those 
based  upon  CSC.  That  is,  the  overall  "batting  average"  for  NPS  selec¬ 
tion  decisions  will  be  improved  by  the  adoption  of  either  set  of  opti¬ 
mal  composites. 


DISCUSSION 

The  Current  NPS  Selection  Composite  employs  weights  for  the  seven 
operational  predictor  variables  which  are  not  optimal.  Therefore,  it 
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is  not  surprising  that  the  regression  composites  of  operational  pre¬ 
dictors:  (1)  had  the  same  or  higher  validity  for  predicting  FGA  and 
S/R  for  NPS  applicants;  and,  (2)  required  information  on  fewer  opera¬ 
tional  predictors.  This  was  to  be  expected. 

The  generally  high  magnitude  of  the  multiple  validity  coefficients 
for  FGA,  particularly  of  the  optimal  selection  composite  employing  the 
operational  and  experimental  predictors,  is  very  encouraging.  These 
multiple  validities,  although  inflated  to  some  degree,  compare  quite 
favorably  to  the  validity  of  CSC.  Moreover,  they  are  substantially 
higher  than  those  found  in  similar  studies.  Lavin  (1965),  in  a  com¬ 
prehensive  review  of  the  literature,  reports  that  the  average  multiple 
validity  of  a  battery  of  predictors  designed  to  forecast  college  grade 
average  is  about  .65  (7.,  p.  52). 

The  multiple  validity  coefficients  for  S/R  are  also  quite  promis¬ 
ing.  Although  likewise  inflated  to  some  degree,  they  compare  favor¬ 
ably  with  the  validity  of  CSC.  The  potential  contribution  of  the  ex¬ 
perimental  predictors  to  NPS  selection  procedures  is  particularly 
evident  for  this  criterion. 

It  should  be  noted  that  the  validities  of  the  various  composites 
for  predicting  S/R  are  considerably  lower  than  the  validities  of  the 
composites  for  predicting  FGA.  This  difference  is  probably  a  function 
of  a  number  of  considerations.  S/R  is  a  more  complex  criterion  than 
is  FGA.  Selection  decisions  for  USNA  applicants  are  based  upon  a  host 
of  different  factors,  some  of  which  (e.g.,  athletic  prowess)  are  beyond 
the  scope  of  the  present  study.  The  statistical  coding  used  to  quanti¬ 
fy  this  measure  of  success  (S/R)  also  contributed  to  the  discrepancy 
(5,  pp.  324-325). 

In  order  to  insure  the  stability  of  the  results,  a  cross-valida¬ 
tion  study  should  be  done.  In  addition,  the  operational  and  experi¬ 
mental  predictors  should  be  validated  against  more  ultimate  criteria 
(e.g.,  USNA  grade  average  and  graduation  vs.  attrition)  as  the  neces¬ 
sary  data  become  available. 

In  view  of  the  demonstrated  potential  utility  of  the  experimental 
predictors  (CBEA,  CBMA,  SATV,  and  SATM)  ,  the  desirability  of  obtain¬ 
ing  these  test  data  for  use  in  future  NPS  selection  decisions  should 
be  assessed  in  terms  of  the  following  considerations:  (1)  the  poten¬ 
tial  reduction  in  the  current  attrition  rate  and  the  concomitant  fi¬ 
nancial  losses  to  the  Navy;  and,  (2)  the  cost  and  administrative  ef¬ 
fort  involved  in  obtaining  this  additional  information  for  all  eligible 
NPS  applicants. 
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APPENDIX  A 

RESULTS  OF  CORRELATION  ANALYSES 


A-l 


Intercorrelations,  Means,  and  Standard  Deviations  of  All  Predictor 
and  Criterion  Variables  in  the  NPS  Applicant  Sample 

(N=246) 
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Intercorrelations,  Means,  and  Standard  Deviations  of  All  Predictor 
and  Criterion  Variables  in  the  NPS  Selectee  Sample 

(N=113) 
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APPENDIX  B 

PROCEDURES  FOR  PREDICTING  CRITERION  SCORES 
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IMPLEMENTATION  OF  A  SELECTION  COMPOSITE 


The  actual  implementation  of  the  alternative  NPS  selection  com¬ 
posites  entails  a  number  of  requisite  steps: 

(1)  Obtaining  and  coding  information  on  an  applicant's  high 
school  record  and  computing  scores  on  the  operational  predictors; 

(2)  Obtaining  the  required  standardized  test  score  information 
on  an  applicant; 

(3)  Forecasting  criterion  performance  for  the  applicant,  using 
his  scores  on  the  predictors  and  one  of  the  setB  of  alternative  com¬ 
posites;  and, 

(4)  Making  an  admissions  decision  (selection  vs.  rejection) 
for  an  NPS  applicant  based  upon  his  predicted  performance  on  the  cri¬ 
teria  and  other  information  considered  pertinent  by  the  NPS  Selection 
Board. 


COMPUTATION  OF  OPERATIONAL  PREDICTOR  SCORES  FROM  HIGH  SCHOOL  RECORD 

In  order  to  provide  a  concrete  illustration  of  the  procedures  in¬ 
volved  in  coding  information  contained  in  a  high  school  transcript, 
the  academic  record  of  a  typical  NPS  applicant  is  presented  in  Table 
B-l. 


Equations  for  computing  the  scores  on  the  operational  predictors 
are  as  follows. 

Major  Subject  Unit  Score 

[1]  MS'J  =  MSU  +  MSU  +  MSU_ 

A  D  U 

This  applicant's  score  would  be: 

MSU  -  1.5  +  2.5  +  2.0 


=  6.0 


The  subscripts  in  the  various  equations  represent  letter  grades 
obtained  in  the  courses. 
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TABLE  B-l 


High  School  Achievement  Data  on  A  Typical  NPS  Applicant 


I 


Optional  Subject  l'nj  t  Score 

f  2  ]  OSU  =  OSl\  +  OSUn  +  OSU„ 

This  applicant's  score  would  be: 
OSU  =  0.5  +  3.5  +  3.0 
c  7.0 


Major  Subject  Quality  Score 

[3]  MSQ  =  4 (MSU  )  +  3(MSU_)  +  2(MSU„) 

A  D  U 

This  applicant’s  score  would  be: 

MSQ  =  4(1.5)  +  3(2.5)  +  2(2.0) 

=  17.5 

Optional  Subject  Quality  Score 

[41  OSQ  =  4(0SU  )  +  2 (OSU  )  +  (OSU„) 

A  D  O 

This  applicant's  score  would  be: 

OSQ  =  4(0.5)  +  2(3.5)  +  (3.0) 

=  12.0 

High  School  Completion  Score 

As  indicated  in  Table  B-l,  this  typical  NPS  applicant  graduated 
from  high  school  and,  therefore,  HSC  =  15.  If  he  had  failed  to  com¬ 
plete  high  school  his  score  would  have  been  aero. 


STANDARDIZED  TEST  RESULTS 

The  test  results  of  this  typical  NPS  applicant  are  presented  in 
Table  B-2. 
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TABLE  B-2 


Standardized  Test  Data  on  A  Typical  NPS  Applicant 


Test  Battery 


A 

Cooperative  School  and  College  Ability  Tests: 
Verbal  Ability: 

Raw  Score  (SVAR) 

Converted  Score  (SVAC.' 

Quantitative  Ability: 

Raw  Score  (SQAR) 

Converted  Score  (SQAC1 

College  Entrance  Examination  Board: 

English  Achievement  (CBEA) 

Mathematics  Achievement  (CBMA) 

Scholastic  Aptitude  Tests: 

Verbal  Aptitude  (SATV) 

Mathematics  Aptitude  (SATM) 


Score 


AO 

305 

40 

318 

558 

631 

551 

627 


* 

The  relationship  between  raw  scores  and  converted  scores  of  the 
Cooperative  School  and  College  Ability  Toe  t  is  explained  elsewhere 
(3.,  £*) .  For  purposes  of  illustration,  :'.t  is  assumed  that  Form  1C  of 
the  SCAT  was  used. 


CURRENT  NPS  SELECTION  COMPOSITE 

Computation  of  the  Current  NPS  Selection  Composite^ 

The  Current  NPS  Selection  Composite  is  computed  using  the  follow¬ 
ing  equation: 

[5]  CSC  -  2(MSU)  +  (OSU)  +  (MSQ)  +  (OSQ)  +  (HSC)  +  (SVAR) 

+  (SQAR) 


2 


Note  that  SCAT  raw  scores  are  used  in  computing  CSC. 
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The  CSC  score  of  this  typical  NPS  applicant  is  obtained  by  substi¬ 
tuting  the  appropriate  values  from  Tables  B-l  and  3-2  into  Equation 
[5]  as  follows: 

CSC  --  2(6.0)  +  (7.0)  +  (17.5)  +  (12.0)  +  (15.0) 

+  (40.0)  +  (40.0) 

-  143.5 


The  weights  applied  to  the  operational  predictors  in  computing 
CSC  were  determined  by  the  NPS  Selection  Board.  These  are  not  the 
most  effective  weights  for  predicting  performance  on  either  of  the 
criteria.  If  it  were  desirable,  for  any  reason,  to  employ  CSC  in 
NPS  selection  decisions,  the  best  available  estimate  of  an  applicant's 
final  grade  average  in  the  NPS  program  would  be: 

[6]  FGA  =  .370  (CSC)  +  33.418 

The  best  prediction  of  FGA  for  this  typical  applicant,  using  his  CSC 
score,  would  be 

FGA  =  .279(143.5)  +  33.418 

=  73.454 

Again,  assuming  the  desirability  of  employing  CSC,  the  best  esti¬ 
mate  of  an  NPS  applicant's  chances  of  being  admitted  to  USNA  is  com¬ 
puted  as  follows: 

S/R  =  .009 (CSC)  -  .649 

For  this  typical  applicant  this  is: 

S/R  «=  .009(143.5)  -  .649 

=  .642 

This  indicates  that  the  best  available  estimate  of  the  probability 
that  this  NPS  applicant  will  be  selected  for  USNA  is  .64,  or  64  in 
100. 
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OPTIMAL  COMPOSITES  OF  OPERATIONAL  PREDICTORS3 

The  optimal  raw  score  weights  for  the  operational  predictors 
whi^h  would  be  used  to  predict  each  of  the  two  criteria  are  presented 
in  Table  B-3. 


TABLE  B-3 

Optimal  Raw  Score  Weights  for  Predicting  NPS  Final  Grade  Average 
and  USNA  Selection  vs.  Rejection  for  NPS  Applicants 
from  Scores  on  the  Operational  Predictors 


Operational  Predictors 

ODtimal  Raw 

FGA 

Score  Weights 
S/R 

MSU 

— 

— 

OSU 

— 

— 

MSQ 

.404 

.019 

OSQ 

.  A08 

— 

use 

— 

— 

SVAC 

— 

— 

SQAC 

.341 

.009 

Additive  Constants: 

-46.111 

-2.631 

Prediction  of  NPS  Final  Grade  Average 

Use  of  the  optimal  composite  for  forecasting  FGA  requires  informa¬ 
tion  on  three  of  the  operational  predictors:  MSQ,  0S0,  and  SQAC.  An 
applicant's  scores  would  be  entered  into  the  following  equation: 

[8]  FGA  =  .404(MSQ)  +  . 408 (OSQ)  +  .341(SQAC)  -  46.111 

Using  the  typical  applicant's  scores,  shown  in  Tables  B-l  and  B-2  above: 

FGA  =  .404(17.5)  +  .408(12.0)  +  .341(318)  -  46.111 

=  74.293 

The  best  prediction  of  this  typical  applicant's  FGA  is  74. 


3 

Note  chat  SCAT  converted  ^  . 
in  all  optimal  composites. 


.-..as,  rather  than  raw  scores,  are  used 
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Prediction  of  USNA  Selection  vs.  Rejection 


The  optimal  composite  for  estimating  S/R  for  an  NPS  applicant 
requires  information  on  only  two  operational  predictors:  MSQ  and 
SQAL.  The  actual  equation  employed  is  as  follows: 

[9]  S/R  =  .019  (MSQ)  +  .009  (SOAC)  -  2.631 

Using  the  scores  of  the  typical  NPS  applicant: 

S/R  .019(17.5)  +  .009(318)  -  2.631 

=  .564 

The  best  available  estimate  of  the  probability  of  this  NPS  applicant 
being  selected  for  USNA  program  is  .56.  His  chances  of  being  accepted 
by  USNA  are  about  56  in  100,  slightly  better  than  fifty-fifty. 


OPTIMAL  COMPOSITES  OF  OPERATIONAL  AND  EXPERIMENTAL  PREDICTORS 

The  optimal  raw  score  weights  for  the  operational  and  experimental 
predictors  which  would  be  used  to  predict  each  of  the  criteria  are 
presented  in  Table  B-4. 

Prediction  of  NPS  Final  Grade  Average 

Use  of  the  optimal  composite  of  operational  and  experimental  pre¬ 
dictors  requires  information  on  five  predictors:  MSQ,  0SQ,  CBEA,  CBMA , 
and,  SATM. 

An  applicant's  scores  would  be  entered  into  the  following  equation: 
[10]  FGA  =  . 202 (MSQ)  +  .349(OSO)  +  .012(CBEA)  +  ,032(CBMA) 

+  .036 (SATM)  +  17.202 

The  best  prediction  of  this  typical  NPS  applicant's  final  grade  average 
is : 


FGA  =  .202(17.5)  +  .349(12.0)  +  .012.(558)  +  .032(631) 

+  .036(627)  +  17.202 
=  74.385 

This  applicant  may  be  expected  to  achieve  a  grade  average  of  about  74 
in  the  NPS  program. 
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TABLE  3-4 


Optimal  Raw  Score  Weights  for  Predicting  NPS  Final  Grade  Average 
and  tJSNA  Selection  vs.  Rejection  for  NPS  Applicants 


from 

and 

Scores  on  the  Operational 
Experimental  Predictors 

Predictor  Variables 

Optimal 

FGA 

Raw  Score  Weights 
S/R 

Operational  Predictors: 

MSU 

— 

— 

OSU 

— 

— 

MSQ 

.202 

— 

OSQ 

.349 

— 

HSC 

— 

— 

SVAC 

— 

SQAC 

— 

— 

Experimental  Predictors: 

CBEA 

.012 

.001 

CBMA 

.032 

— 

SATV 

— 

— 

SATM 

.036 

.003 

Additive  Const  ts: 

17.202 

-1.772 

Prediction  of  USNA  Selection  vs.  Rejection 

The  optimal  composite  of  operational  and  experimental  predictors 
requires  no  information  on  the  operational  predictors  and  only  include: 


two  of  the  four  experimental  predictors:  CBEA  and  SATM.  These  test 
scores  are  entered  into  the  following  equation: 

(11)  S/R  =  .OOl(CBEA)  +  .003 (SATM)  -  1.772 

Substituting  this  typical  applicant's  test  scores  into  this  equation 
gives  : 

S/R  =  .001(558)  +  .003(627)  -  1.772 
=  .667 

The  best  available  estimate  of  this  applicant's  probability  of  being 
selected  for  USNA  is  .67,  or  about  two  chances  in  three. 
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